{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "http://cs229.stanford.edu/ps/ps2/ps2.pdf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# (a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Theory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For Navie Bayes represented as a multinomial event model, to maximize its maxmimum likelihood with Laplace smoothing,\n",
    "\n",
    "\\begin{align*}\n",
    "\\phi_{k|y=1} &= \\frac{\\sum_{i=1}^{m} \\sum_{j=1}^{n_i} \\boldsymbol{1} \\{ x_j^{(i)} = k \\wedge y^{(i)} =1 \\} + 1}{\\sum_{i=1}^{m} \\boldsymbol{1} \\{ y^{(i)} = 1 \\} n_i + \\left| V \\right|} \\\\\n",
    "\\phi_{k|y=0} &= \\frac{\\sum_{i=1}^{m} \\sum_{j=1}^{n_i} \\boldsymbol{1} \\{ x_j^{(i)} = k \\wedge y^{(i)} =0 \\} + 1}{\\sum_{i=1}^{m} \\boldsymbol{1} \\{ y^{(i)} = 0 \\} n_i + \\left| V \\right|} \\\\\n",
    "\\phi_{y} &= \\frac{\\sum_{i=1}^{m} \\boldsymbol{1} \\{ y^{(i)} = 1 \\}}{m}\n",
    "\\end{align*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For prediction,\n",
    "\n",
    "\\begin{align*}\n",
    "p(y=1|x) &= \\frac{p(x|y=1)p(y=1)}{p(x)} \\\\\n",
    "              &= \\frac{p(x|y=1)p(y=1)}{p(x|y=1)p(y=1) + p(x|y=0) p(y=0)} \\\\\n",
    "              &= \\frac{1}{1 + \\frac{p(x|y=0) p(y=0)}{p(x|y=1) p(y=1)}} \\\\\n",
    "              &= \\frac{1}{1 + \\mathrm{exp} (\\mathrm{log}P(x|y=0) + \\mathrm{log}P(y=0) - \\mathrm{log}P(x|y=1) - \\mathrm{log} P(y=1))}\n",
    "\\end{align*}\n",
    "\n",
    "The last two equalities are for numerical stability"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See implementation in `nb.py`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# (b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import glob\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import nb\n",
    "\n",
    "%matplotlib inline\n",
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "trainMatrix, tokenlist, trainCategory = nb.readMatrix('spam_data/MATRIX.TRAIN')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "state = nb.nb_train(trainMatrix, trainCategory)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['phi_yeq1', 'phi_yeq0', 'phi'])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "state.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['httpaddr', 'spam', 'unsubscrib', 'ebai', 'valet', 'diploma', 'dvd',\n",
       "       'websit', 'click', 'lowest', 'numberpx', 'arial', 'helvetica',\n",
       "       'serif', 'nashua', 'amherst', 'mortgag', 'refin', 'charset',\n",
       "       'newslett', 'customerservic', 'numberpt', 'iso', 'web', 'lender',\n",
       "       'numberd', 'loan', 'dailybargainmail', 'coral', 'html', 'unsolicit',\n",
       "       'www', 'fl', 'holidai', 'equiti', 'tal', 'consolid', 'bachelor',\n",
       "       'sweepstak', 'subscript', 'mba', 'bonu', 'ae', 'refinanc', 'hover',\n",
       "       'mime', 'untitl', 'subscrib', 'recur', 'postal'],\n",
       "      dtype='<U16')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokens = np.array(tokenlist)\n",
    "tokens[np.argsort(state['phi_yeq1'] / state['phi_yeq0'])[::-1]][:50]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These are the top words most indicative of spam emails, kind of makes sense."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# (c)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error: 0.0387\n",
      "Error: 0.0262\n",
      "Error: 0.0262\n",
      "Error: 0.0187\n",
      "Error: 0.0175\n",
      "Error: 0.0163\n"
     ]
    }
   ],
   "source": [
    "files = sorted(glob.glob('./spam_data/MATRIX.TRAIN.[0-9]*'), key=lambda s: int(s.rsplit('.')[-1]))\n",
    "\n",
    "nb_sizes = []\n",
    "nb_errs = []\n",
    "mat_test, tok_test, cat_test = nb.readMatrix('./spam_data/MATRIX.TEST')\n",
    "for f in files:\n",
    "    mat, tok, cat = nb.readMatrix(f)\n",
    "    nb_sizes.append(mat.shape[0])\n",
    "    mod = nb.nb_train(mat, cat)\n",
    "    output = nb.nb_test(mat_test, mod)\n",
    "    nb_errs.append(nb.evaluate(output, cat_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x11334b080>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuYXXV97/H3Z657JplbkskmJGBA4iWghJiHRq0tRbEJ\nrURapHpKxZ72IEet93pQ+3jQ3rVgH7xgvRarrUWlih6qIMXHFgUNGmJCQCJCScw9k9tkMrd8zx9r\n7cmeyU5mz+zZs/fM/ryeZz977bV+a893QTKfrN9vrfVTRGBmZjZZdZUuwMzMZjYHiZmZlcRBYmZm\nJXGQmJlZSRwkZmZWEgeJmZmVxEFiZmYlcZCYmVlJHCRmZlaShkoXMB0WLFgQS5curXQZZmYzykMP\nPbQ3IrrHa1cTQbJ06VLWr19f6TLMzGYUSU8V085dW2ZmVhIHiZmZlcRBYmZmJXGQmJlZSRwkZmZW\nEgeJmZmVxEFiZmYlcZCcxr1bdvHx726tdBlmZlXNQXIa/7V1Lx+/7+eVLsPMrKo5SE4j257hSP8Q\nR/qHKl2KmVnVcpCcRra9GYBdh45VuBIzs+rlIDmNbHsGcJCYmZ2Og+Q0ckGy+1B/hSsxM6teDpLT\n8BmJmdn4HCSnMbe5gTlN9ex0kJiZnZKDZBzZjoy7tszMTsNBMo5sW8ZdW2Zmp+EgGUe2vdldW2Zm\np+EgGUeuaysiKl2KmVlVcpCMI9uWYWD4OAeODla6FDOzquQgGcfIJcCH3b1lZlaIg2Qcucek7Dzo\nIDEzK8RBMg7f3W5mdnoOknEs9IMbzcxOy0EyjuaGerpaGz1GYmZ2Cg6SImTbM+w86K4tM7NCHCRF\nyLZn2O0zEjOzghwkRci2N3uMxMzsFBwkRci2Z9hzuJ+h4eOVLsXMrOo4SIqQbc9wPGBf70ClSzEz\nqzoOkiJ4giszs1NzkBQhO3Ivia/cMjMby0FShDPSMxI/Tt7M7GQOkiLMn9tMnWC3g8TM7CQOkiLU\n14nuNl8CbGZWiIOkSNn2DDs9RmJmdhIHSZGy7Rl3bZmZFeAgKZLvbjczK6ysQSJpjaTHJG2VdEOB\n7ZJ0S7p9o6SV6fqMpB9KeljSZknvz9vnRknbJW1IX5eX8xhysm0Zeo4O0j80PB0/zsxsxihbkEiq\nBz4GrAWWA6+RtHxMs7XAsvR1HXBrur4fuDQiLgRWAGskrc7b78MRsSJ93VWuY8jnCa7MzAor5xnJ\nxcDWiHgiIgaALwHrxrRZB3w+Eg8AnZIWpZ+PpG0a01eUsdZxZTt8d7uZWSHlDJLFwNN5n7el64pq\nI6le0gZgN3BPRDyY1+5P0q6wz0rqKvTDJV0nab2k9Xv27Cn1WHx3u5nZKVTtYHtEDEfECmAJcLGk\nC9JNtwLnknR57QBuOsX+n4yIVRGxqru7u+R6sm2+u93MrJByBsl24Ky8z0vSdRNqExEHgPuANenn\nXWnIHAc+RdKFVnadrY00NdT5EmAzszHKGSQ/ApZJOkdSE/Bq4M4xbe4EXptevbUaOBgROyR1S+oE\nkNQCXAY8mn5elLf/lcCmMh7DCEm+BNjMrICGcn1xRAxJehPwbaAe+GxEbJZ0fbr9E8BdwOXAVuAo\n8Ifp7ouA29Irv+qA2yPim+m2D0paQTL4/iTw+nIdw1jZtozHSMzMxihbkACkl+beNWbdJ/KWA3hj\ngf02Ahed4jv/YIrLLFq2PcOWHYcq9ePNzKpS1Q62V6Nse8ZdW2ZmYzhIJiDb3kzvwDBH+ocqXYqZ\nWdVwkEyAp9w1MzuZg2QCRoLkoIPEzCzHQTIBI3e3H3aQmJnlOEgmYOFI15YvATYzy3GQTMDc5gbm\nNjew011bZmYjHCQTlG1vZre7tszMRjhIJii5l8RdW2ZmOQ6SCfJNiWZmozlIJijbnmH3oX6Sp7uY\nmZmDZIKy7c0MDB+n5+hgpUsxM6sKDpIJ8t3tZmajOUgmKHdTomdKNDNLOEgmKHdG4pkSzcwSDpIJ\n6m5LH5PiS4DNzAAHyYQ1N9Qzb06Tx0jMzFIOkknwvSRmZic4SCYh297sri0zs5SDZBKybT4jMTPL\ncZBMQra9mb1H+hkaPl7pUszMKs5BMgnZjgzHA/YeGah0KWZmFecgmYRsm+9uNzPLcZBMgh+TYmZ2\ngoNkErIduZsSHSRmZg6SSZg/p5n6OvkSYDMzHCSTUl8nuuc2+4zEzAwHyaRl25vZddhnJGZmDpJJ\nyrZn2HXQZyRmZg6SScq2Z9h12EFiZuYgmaRsezMHjg5ybHC40qWYmVWUg2SSTkxw5XESM6ttDpJJ\nGrkp0d1bZlbjHCST5LvbzcwSZQ0SSWskPSZpq6QbCmyXpFvS7RslrUzXZyT9UNLDkjZLen/ePvMk\n3SPp8fS9q5zHcCrZdk+5a2YGZQwSSfXAx4C1wHLgNZKWj2m2FliWvq4Dbk3X9wOXRsSFwApgjaTV\n6bYbgHsjYhlwb/p52nW0NNLcUOczEjOreeU8I7kY2BoRT0TEAPAlYN2YNuuAz0fiAaBT0qL085G0\nTWP6irx9bkuXbwNeWcZjOCVJnnLXzIzyBsli4Om8z9vSdUW1kVQvaQOwG7gnIh5M22QjYke6vBPI\nTnXhxUqm3HWQmFltq9rB9ogYjogVwBLgYkkXFGgTnDhTGUXSdZLWS1q/Z8+estSYnJF4jMTMals5\ng2Q7cFbe5yXpugm1iYgDwH3AmnTVLkmLANL33YV+eER8MiJWRcSq7u7uSR/E6eS6tpI8MzOrTeUM\nkh8ByySdI6kJeDVw55g2dwKvTa/eWg0cjIgdkroldQJIagEuAx7N2+fadPla4OtlPIbTyrY3c3Rg\nmCP9Q5Uqwcys4hrK9cURMSTpTcC3gXrgsxGxWdL16fZPAHcBlwNbgaPAH6a7LwJuS6/8qgNuj4hv\nptv+Brhd0h8BTwFXl+sYxnPiXpJ+2jKNlSrDzKyiyhYkABFxF0lY5K/7RN5yAG8ssN9G4KJTfOc+\n4KVTW+nk5N+UeN7CuRWuxsysMqp2sH0m8N3tZmYOkpL47nYzMwdJSVqbGmjLNPiMxMxqmoOkRL67\n3cxqnYOkRL673cxqnYOkRNk2391uZrXNQVKibEeG3YePcfy47243s9o0bpCkD09823QUMxNl25oZ\nHA56jg5UuhQzs4oYN0giYhh4zTTUMiPl391uZlaLiu3aul/SRyW9RNLK3Kuslc0Q2Q7flGhmta3Y\nR6SsSN8/kLcugEuntpyZx3e3m1mtKypIIuI3yl3ITNU913e3m1ltK6prS1KHpJtzE0VJuklSR7mL\nmwmaGuqYP6eJXYd9RmJmtanYMZLPAodJHtl+NXAI+Fy5ipppsu0Zdh10kJhZbSp2jOSZEfG7eZ/f\nn86nbqR3t/uMxMxqVLFnJH2SfjX3QdKLgb7ylDTzeO52M6tlxZ6RXA98Pm9cpIcT093WvGx7hr1H\n+hkaPk5DvR8WYGa1ZdwgkVQHPDsiLpTUDhARh8pe2QySbc8QAXuO9LOoo6XS5ZiZTati7mw/Drwr\nXT7kEDmZJ7gys1pWbD/MdyS9U9JZkublXmWtbAbxTYlmVsuKHSP5vfT9jXnrAjh3asuZmRwkZlbL\nih0juSYi7p+Gemak+XOaqK+Tg8TMalKxYyQfnYZaZqy6OrGwrdljJGZWk4odI7lX0u9KUlmrmcEW\neu52M6tRxQbJ64HbgX5JhyQdluSrt/Kc4bnbzaxGFRskHcDrgL+IiHbgfOCychU1E/nudjOrVcUG\nyceA1ZyYKfEwHjcZJdue4WDfIMcGhytdipnZtCr28t9fiYiVkn4CEBE9kprKWNeMk7sE+Lp/eojm\nhql/TMraC87gd1YumfLvNTMrVbFBMiipnuTeESR1A8fLVtUMdPHSeVx0did7Dk9999ahvkH+49Hd\nnNs9lxVndU7595uZlUIRMX4j6fdJbkpcCdwGXAX8WUR8ubzlTY1Vq1bF+vXrK13GpB3sG2Tt33+P\nTFM9d735JWQa6ytdkpnVAEkPRcSq8doV1QcTEV8ked7WXwM7gFfOlBCZDTpaGvnQqy7kiT29/O23\nHq10OWZmoxTbtUVEPAr4t1iFvPi8BVz7wmfwufuf5LLnZnnReQsqXZKZGVD8VVtWBW5Y+1zOWTCH\nP/3KRg4dG6x0OWZmgINkRmlpquemqy9kx8E+/vwbj1S6HDMzwEEy46w8u4vrf/2ZfPmhbXznkV2V\nLsfMrLxBImmNpMckbZV0Q4HtknRLun2jpJXp+rMk3SfpEUmbJb0lb58bJW2XtCF9XV7OY6hGb33Z\ns3juonZuuGMj+474bnozq6yyBUl638nHgLXAcuA1kpaPabYWWJa+rgNuTdcPAe+IiOUkd9S/ccy+\nH46IFenrrnIdQ7Vqaqjj5qsv5GDfIH/2tU0Ucwm3mVm5lPOM5GJga0Q8EREDwJeAdWParAM+H4kH\ngE5JiyJiR0T8GCAiDgNbgMVlrHXGee6idt522bP49007+fqGX1a6HDOrYeUMksXA03mft3FyGIzb\nRtJS4CLgwbzVf5J2hX1WUtdUFTzTvP7XnsnKszt539c3sfOgnzxsZpVR1YPtkuYCXwXeGhG5x9bf\nSjLF7wqSmyNvOsW+10laL2n9nj17pqXe6VZfJ266egWDw8G7vrrRXVxmVhHlDJLtwFl5n5ek64pq\nI6mRJES+GBF35BpExK6IGE5nbvwUSRfaSSLikxGxKiJWdXd3l3ww1eqcBXN4z+XP4Xs/28MXH/zv\nSpdjZjWonEHyI2CZpHPSJwW/GrhzTJs7gdemV2+tBg5GxI50JsbPAFsi4ub8HSQtyvt4JbCpfIcw\nM1yz+hm8ZNkC/vL/beHJvb2VLsfMakzZgiQihoA3Ad8mGSy/PSI2S7pe0vVps7uAJ4CtJGcXb0jX\nvxj4A+DSApf5flDSTyVtBH4DeFu5jmGmkMQHr3o+DfXinV9+mOHj7uIys+lT1NN/Z7qZ/vTfYt3x\n4228/faHuWHtc7j+159Z6XLMbIab0qf/2sxw5UWL+c3zs9x89894dOeh8XcwM5sCDpJZRBJ/deXz\naG9p4O3/+jADQ557zMzKz0Eyy8yf28xfXfk8HtlxiI/8x+OVLsfMaoCDZBZ6+fln8Lsrl/Cx+7by\nk//uqXQ5ZjbLOUhmqf97xXLOaM/wjtsfpm9guNLlmNks5iCZpdoz6fS8ez09r5mVl4NkFnvxeQt4\n3YuW8o/ff5Lvb91b6XLMbJZykMxy/2fNczjX0/OaWRk5SGa5lqZ6/s7T85pZGTlIasDKs7v435ck\n0/Pe4+l5zWyKOUhqxFtemkzP+25Pz2tmU8xBUiNy0/Me6hvy9LxmNqUcJDXE0/OaWTk4SGrMdb92\nLi94Rpen5zWzKeMgqTH1deKmV13I4HDwp1952F1cZlYyB0kNWppOz/ufj+/lC56e18xK5CCpUbnp\nef/K0/OaWYkcJDXK0/Oa2VRxkNSwRR0tfGDd+ax/qodP/ecTlS7HzGYoB0mNe+WKxaw5/wxPz2tm\nk+YgqXGS+MsrL6C9pYG3eXpeM5sEB4mNTM+7ZcchbrnX0/Oa2cQ4SAxIpue96gVL+Ph3PT2vmU2M\ng8RGvO8Vy1nU0eLpec1sQhwkNqI908iHrnq+p+c1swlxkNgoL/L0vGY2QQ4SO0luet53fvlhT89r\nZuNykNhJWprquenqC9l56Bgf8PS8ZjYOB4kVdNHZXbzhkvP4iqfnNbNxOEjslN780mUs9/S8ZjYO\nB4mdUlNDHTf/XjI973v/zdPzmllhDhI7reeckUzP+63NO/nahu2VLsfMqpCDxMZ1Ynrezew42Ffp\ncsysyjhIbFy56XmHhoN3fWWju7jMbBQHiRVl6YI5vOe3nuvpec3sJGUNEklrJD0maaukGwpsl6Rb\n0u0bJa1M158l6T5Jj0jaLOktefvMk3SPpMfT965yHoOdcM2vnO3pec3sJGULEkn1wMeAtcBy4DWS\nlo9pthZYlr6uA25N1w8B74iI5cBq4I15+94A3BsRy4B70882DfKn532Hp+c1s1Q5z0guBrZGxBMR\nMQB8CVg3ps064POReADolLQoInZExI8BIuIwsAVYnLfPbenybcAry3gMNsaijhb+fN0FPPRUD5/8\nnqfnNbPyBsli4Om8z9s4EQZFt5G0FLgIeDBdlY2IHenyTiA7NeVasdatOJO1F5zBh+/x9LxmVuWD\n7ZLmAl8F3hoRJ/3GiuTyoYL9K5Kuk7Re0vo9e/aUudLaIom/eKWn5zWzRDmDZDtwVt7nJem6otpI\naiQJkS9GxB15bXZJWpS2WQTsLvTDI+KTEbEqIlZ1d3eXdCB2svlzm/nr33m+p+c1s7IGyY+AZZLO\nkdQEvBq4c0ybO4HXpldvrQYORsQOSQI+A2yJiJsL7HNtunwt8PXyHYKdzmXLs56e18zKFyQRMQS8\nCfg2yWD57RGxWdL1kq5Pm90FPAFsBT4FvCFd/2LgD4BLJW1IX5en2/4GuEzS48DL0s9WIZ6e18xU\nC3cpr1q1KtavX1/pMmat72/dy//49IO87kVLufGK8ytdjplNEUkPRcSq8dpV9WC7zQz50/Pe7+l5\nzWqOg8SmRG563j/19LxmNcdBYlPC0/Oa1S4HiU2Z/Ol57968s9LlmNk0cZDYlMpNz/uef/upp+c1\nqxENlS7AZpemhjo+/HsreMVH/otX/cMPWLGkk8VdLSzpamFxZyuLu1o4szNDc0N9pUs1syniILEp\n9+wz2vi7qy/kCz94igd/sZ8dG/oY+6Dg7rZmFnemAdPVwpLO5D0XNnOb/UfTbKbw31YriysuPJMr\nLjwTgKHh4+w8dIxtPX1s7+lj+4ET75u2H+TuzbsYGB79vK7O1kYWd7Ykr66WkdBZ0tXK4s4WOlsb\nSR6AYGaV5iCxsmuor2NJVytLuloLbj9+PNh7pJ9teQGzreco23v6eHJfL/dv3UvvmLvmW5vqx4RM\n66jA6Z7bTF2dg8ZsOjhIrOLq6sTC9gwL2zOsPPvkCS8jgoN9g2zr6UvOakYC5yjbD/Sx4ekDHDg6\n+t6Vpvo6FnVm0rGZE11muaA5oyNDY72vNTGbCg4Sq3qS6GxtorO1iQsWdxRs09s/NBIwuTObbT1J\n0Hz3sT3sPjz6CrI6wRntmZFwyY3P5MZsFne2kGn0BQFmxXCQ2Kwwp7mBZ2XbeFa2reD2/qFhdhxI\nx2kOHB0VOOuf6uEbG3ecNHXwgrlNJ3WbjYROVwvtmcbpODSzqucgsZrQ3FDP0gVzWLpgTsHtQ8PH\n2XW4/0SX2chYTR9bdhziO1t20T9mAq+2TMPI4P+SrpMvDJg3p8kXBFhNcJCYkVwQkDvjgHknbY8I\n9h4ZONF9lnab5ZYffGIfh/uHRu2TaUy/My9s8gNnYVuGel8QYLOAg8SsCJLobmumu62ZFWd1FmyT\nXBBw9KRLnLf1JJc57+8dGNW+sV6c0ZFhSd6FAPn31CzqaKGpwRcEWPVzkJhNkY6WRjpaOjj/zMIX\nBBwdGOKXB05ceZZ/X81/Pb6XXYePkT89kATZtrEXBOSf2bTS0uQLAqzyHCRm06S1qYHzFrZx3sLC\nFwQMDB1n58FjbOs5Ouqemu09ffzk6R7u+ukOhsZcEDBvThOLOjLMm9PE/DlNzJvTzPy5TXS1NiXr\n5jaNbGvPNPreGisLB4lZlWhqqOPs+a2cPb/wjZvDx4Pdh4+N6jLb1tPHrkPH2Nc7wJP7etl/ZOCk\nmzdz6utEV2sj8+bkwqWZrjmNSfjMORE4XXnvvtfGiuEgMZsh6uvEoo5k7OR0c58eGxym5+gA+44M\nsL83ee3rHaAnfd/f28/+3gEe3XmI/b0DHOgb5FQzbrdnGkaCZyRw5jYxLz3jmTc3DZ3W5Oyntcm/\nUmqR/6+bzTKZxvqRwCnG0PBxDvQNjoROLnj2HxlIAikNn209R9m47QA9RwcYHC6cPJnGOubPaWZe\n3pnNvAJnO7kzorZMg7vbZgEHiVmNa6ivY8HcZhbMbS6qfURwuH+I/UdyITP6bCd39rO/d4An9hxh\nf+8AR0/b3ZYXOHlnO7nxnVGv1iYa3N1WdRwkZjYhkmjPNNKeaTzlDZ5jHRscHn2209vP/t7BkW62\nXDfclh1pd9uYZ6fl62hpHBUu88eGTXq2kwslX9lWfg4SMyu7TGM9Z3a2cGZn8d1tPUcHx4z19I8Z\n6xng6f1HefjpA+zvHTjpiraclsb60Wc4pxjfmZd2ybVnGvxEgglykJhZ1Wmorxu5AZTs+O0jgkPH\nhkYCJ3e2kxvr2d87wP6jyfvju5Lutr7Bwt1tDXUaZ3ynedTZT1drY813tzlIzGzGk5TeENrIOUV2\nt/UNDCfhcmSAfWkX29gLDnp6B3jkl4fY1zvAwb7C3W1SXnfbSeM7zcwbc4n1vDlNs+7J0g4SM6tJ\nLU31LG7KPV9tfIPDx+k5OkBP7+Co4NmXf3XbkQGe2neUnzx9gJ7TdLe1NtWPGt/JP9s5ad3cJtqa\nq7u7zUFiZlaExvo6FrZlWNiWAQo/nSBfRHCob+iks52xV7rtPTLAz3YdYV9vP8cGjxf8rsZ6jXla\nQfPI+E5urCe/C66rtWlaHwjqIDEzKwNJdLQ20tHayLndxe3TNzA8Ejz5l1Hnznb29SZnP5u2H2Tf\nkX4OHRsq+D0SdKbdbX955fNYfe78KTyykzlIzMyqREtTPUuaWlnSVfgxOWPlutv25wXN2LGejpby\nT8DmIDEzm6FGd7dVTm1fs2ZmZiVzkJiZWUkcJGZmVhIHiZmZlcRBYmZmJXGQmJlZSRwkZmZWEgeJ\nmZmVRHGqyZpnEUl7gKfyVi0A9laonMlyzdPDNU8P1zx9Sqn7GREx7gNeaiJIxpK0PiJWVbqOiXDN\n08M1Tw/XPH2mo253bZmZWUkcJGZmVpJaDZJPVrqASXDN08M1Tw/XPH3KXndNjpGYmdnUqdUzEjMz\nmyI1FSSS1kh6TNJWSTdUup4cSWdJuk/SI5I2S3pLun6epHskPZ6+d+Xt8+70OB6T9JsVrL1e0k8k\nfXMm1CypU9JXJD0qaYukF86Amt+W/rnYJOlfJGWqsWZJn5W0W9KmvHUTrlPSCyT9NN12i8o4Wfkp\nav5Q+udjo6R/k9RZ7TXnbXuHpJC0YFprjoiaeAH1wM+Bc4Em4GFgeaXrSmtbBKxMl9uAnwHLgQ8C\nN6TrbwD+Nl1entbfDJyTHld9hWp/O/DPwDfTz1VdM3Ab8MfpchPQWc01A4uBXwAt6efbgddVY83A\nrwErgU156yZcJ/BDYDUg4N+BtdNc88uBhnT5b2dCzen6s4Bvk9wzt2A6a66lM5KLga0R8UREDABf\nAtZVuCYAImJHRPw4XT4MbCH5BbKO5Bcf6fsr0+V1wJcioj8ifgFsJTm+aSVpCfBbwKfzVldtzZI6\nSP4SfgYgIgYi4kA115xqAFokNQCtwC+pwpoj4nvA/jGrJ1SnpEVAe0Q8EMlvu8/n7TMtNUfE3RGR\nmwz9AWBJtdec+jDwLiB/4Htaaq6lIFkMPJ33eVu6rqpIWgpcBDwIZCNiR7ppJ5BNl6vlWP6e5A/u\n8bx11VzzOcAe4HNpd9ynJc2himuOiO3A3wH/DewADkbE3VRxzWNMtM7F6fLY9ZXyP0n+tQ5VXLOk\ndcD2iHh4zKZpqbmWgqTqSZoLfBV4a0Qcyt+W/quhai6xk/TbwO6IeOhUbaqtZpJ/2a8Ebo2Ii4Be\nku6WEdVWczqmsI4kBM8E5ki6Jr9NtdV8KjOlzhxJ7wWGgC9WupbTkdQKvAd4X6VqqKUg2U7Sh5iz\nJF1XFSQ1koTIFyPijnT1rvQUlPR9d7q+Go7lxcAVkp4k6Sa8VNIXqO6atwHbIuLB9PNXSIKlmmt+\nGfCLiNgTEYPAHcCLqO6a8020zu2c6ErKXz+tJL0O+G3g99MAhOqt+Zkk/9B4OP37uAT4saQzmKaa\naylIfgQsk3SOpCbg1cCdFa4JgPRqic8AWyLi5rxNdwLXpsvXAl/PW/9qSc2SzgGWkQycTZuIeHdE\nLImIpST/Lf8jIq6p8pp3Ak9Lena66qXAI1RxzSRdWqsltaZ/Tl5KMoZWzTXnm1CdaTfYIUmr0+N9\nbd4+00LSGpIu2ysi4mjepqqsOSJ+GhELI2Jp+vdxG8nFOzunreZyXVlQjS/gcpIron4OvLfS9eTV\n9askp/wbgQ3p63JgPnAv8DjwHWBe3j7vTY/jMcp4hUiR9V/Ciau2qrpmYAWwPv1v/TWgawbU/H7g\nUWAT8E8kV+BUXc3Av5CM4wyS/DL7o8nUCaxKj/XnwEdJb5yexpq3kowr5P4ufqLaax6z/UnSq7am\nq2bf2W5mZiWppa4tMzMrAweJmZmVxEFiZmYlcZCYmVlJHCRmZlYSB4nNOkqe8PuGSe57V/7TXk/R\n5gOSXja56k77va+T9NFx2lwi6UVT9PO+PxXfY+YgsdmoEygYJOmDD08pIi6P5EGOp2vzvoj4Tgn1\nleISkjvbSxYRU/I9Zg4Sm43+BnimpA3p3BKXSPpPSXeS3MmOpK9JekjJPB/X5XaU9KSkBZKWKpmv\n5FNpm7sltaRt/lHSVXnt3y/px+ncDs9J13crmX9jc/pwyKfy54jI+3l/KOlnkn5I8tiZ3PpXSHow\nfbjkdyRl0wd6Xg+8LT22lxRqV+BnnC/ph+k+GyUtS9cfSd8/kG7bIGm7pM+l66/J2+8fJNVPyf8d\nm32m665Xv/yarhewlNHzS1xC8oDGc/LWzUvfW0ju7p2ffn4SWJB+xxCwIl1/O3BNuvyPwFV57f8k\nXX4D8Ol0+aPAu9PlNSRPLlgwps5FJI9A6SaZG+V+4KPpti5OTIX9x8BN6fKNwDvzvqNguzE/5yMk\nz4wi/Tm5uU2OjGnXCfwUeAHwXOAbQGO67ePAayv9/9av6nyd9jTfbBb5YSTzMeS8WdKV6fJZJM8g\n2jdmn19ExIZ0+SGScCnkjrw2v5Mu/ypwJUBEfEtST4H9fgX4bkTsAZD0r8Cz0m1LgH9NH3TYRDK5\nVSHFtPsB8F4l88fcERGPj22QPm/pC8DNEfGQpDeRBMqPkk20cOKBi2ajuGvLakVvbkHSJSRP1X1h\nRFwI/ASkQNvgAAABbElEQVTIFNinP295GE75D6/+ItpM1EdIzk6eB7z+FPUV1S4i/hm4AugD7pJ0\naYHvuZHkycifSz8LuC0iVqSvZ0fEjaUckM1eDhKbjQ6TTFl8Kh1AT0QcTcc0VpehhvuBqwEkvZyk\nC2qsB4FflzRfyTQCrxpTY+6x3tfmrR97bKdqN0LSucATEXELyRNenz9m+ytIgvXNeavvBa6StDBt\nM0/SMwofqtU6B4nNOhGxD7hf0iZJHyrQ5FtAg6QtJAPzD5ShjPcDL5e0iSQgdpKEQH6dO0jOBH5A\nEjxb8jbfCHxZ0kPA3rz13wCuzA22n6ZdvquBTZI2ABeQTKua7+0ks+PlBtY/EBGPAH8G3C1pI3AP\nyZiO2Un89F+zMpDUDAxHxJCkF5LMyrii0nWZlYMH283K42zgdkl1wADwvypcj1nZ+IzEzMxK4jES\nMzMriYPEzMxK4iAxM7OSOEjMzKwkDhIzMyuJg8TMzEry/wEUpZvPkt9psAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10ca53ac8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(nb_sizes, nb_errs)\n",
    "plt.xlabel('training set size')\n",
    "plt.ylabel('error')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# (d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import svm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error: 0.0225\n",
      "Error: 0.0200\n",
      "Error: 0.0025\n",
      "Error: 0.0037\n",
      "Error: 0.0000\n",
      "Error: 0.0000\n"
     ]
    }
   ],
   "source": [
    "files = sorted(glob.glob('./spam_data/MATRIX.TRAIN.[0-9]*'), key=lambda s: int(s.rsplit('.')[-1]))\n",
    "\n",
    "svm_sizes = []\n",
    "svm_errs = []\n",
    "mat_test, tok_test, cat_test = svm.readMatrix('./spam_data/MATRIX.TEST')\n",
    "for f in files:\n",
    "    mat, tok, cat = svm.readMatrix(f)\n",
    "    svm_sizes.append(mat.shape[0])\n",
    "    mod = svm.svm_train(mat, cat)\n",
    "    output = svm.svm_test(mat_test, mod)\n",
    "    svm_errs.append(svm.evaluate(output, cat_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x1126e72e8>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X+cFfV97/HXZ38Bu/zcPQvCLngOSlRMFAgBmphUozFq\nogj2Wr1JNam9xFtN26R5tLbpzU3yuLc3Ta9Ja021JjHRNo2xESKxNIbYpEm9UYHlhyCCCIvs8mth\nV36zy+5+7h8ziyebBQ7szpk557yfj8d57JyZ75z9DD/2vTPzne/X3B0REZFzVRZ3ASIiUtgUJCIi\nMigKEhERGRQFiYiIDIqCREREBkVBIiIig6IgERGRQVGQiIjIoChIRERkUCriLiAfUqmUp9PpuMsQ\nESkoq1at2ufu9WdqVxJBkk6nWblyZdxliIgUFDPbnks7XdoSEZFBUZCIiMigKEhERGRQFCQiIjIo\nChIRERkUBYmIiAyKgkRERAZFQXIav3x9Pw/97PW4yxARSTQFyWn8dNNe/vrZV9my93DcpYiIJJaC\n5DQ+8b6pDK8s529+sjnuUkREEktBchp1I4fx8fekeWbdLl7dfTDuckREEklBcgaL3nsBo4ZX8NXl\nOisRERmIguQMxlRX8ntXTOXZDXt4ueVA3OWIiCSOgiQHv3tFmrHVlXxl+aa4SxERSRwFSQ5GDa/k\nE++7gJ9uamPV9o64yxERSRQFSY7ufPf5pEZW6axERKQfBUmOqqsq+O9XXsjzW/bzy9f3x12OiEhi\nKEjOwkfmTmHC6GF8Zfkm3D3uckREEkFBchaGV5Zz7/unsaK5g1+8ti/uckREEkFBcpZ+e/ZkGsaO\n4P4f66xERAQUJGetqqKMP7j6Qta2HOC5jXvjLkdEJHYKknOwcFYj6bpq7l++md5enZWISGlTkJyD\nyvIy/vCaaWzcdZAfbdgddzkiIrFSkJyjmy5v4MLxI/nq8s306KxEREqYguQclZcZn7rmbby29zA/\nXLsz7nJERGKjIBmE699+HhefN4q/+clmunt64y5HRCQWCpJBKCsz/vjai2jef5TFTa1xlyMiEotI\ng8TMrjOzTWa2xczuG2C7mdkD4fZ1ZjYrXD/ZzH5qZq+Y2QYz+8OsfWrNbLmZvRZ+HRflMZzJNZeM\n5/LGMfztc6/R1a2zEhEpPZEFiZmVA18DrgemA7eb2fR+za4HpoWvRcBD4fpu4I/dfTowD7gna9/7\ngOfcfRrwXPg+NmbGp6+9iNY3j/G9lTviLEVEJBZRnpHMAba4+1Z37wKeAOb3azMfeNwDLwBjzWyi\nu+9y9yYAdz8EbAQasvZ5LFx+DLg5wmPIyfumpZh9/jge/PfXOH6iJ+5yRETyKsogaQCyf0Vv4a0w\nyLmNmaWBmcCL4aoJ7r4rXN4NTBjom5vZIjNbaWYr29razqX+nJkZi943lT0HO1m7481Iv5eISNIk\n+ma7mY0EngL+yN0P9t/uwWBXAz7E4e6PuPtsd59dX18fcaVwycTRADTvPxL59xIRSZIog6QVmJz1\nvjFcl1MbM6skCJHvuPvirDZ7zGxi2GYikIgBryaNHUFVeRlb9ylIRKS0RBkkK4BpZpYxsyrgNmBp\nvzZLgTvC3lvzgAPuvsvMDPgmsNHdvzLAPneGy3cCT0d3CLkrLzOm1FXTrCARkRJTEdUHu3u3md0L\nPAuUA4+6+wYzuzvc/jCwDLgB2AIcBT4e7v4e4HeAl81sTbjuz919GfAl4EkzuwvYDtwa1TGcrUyq\nhm0KEhEpMZEFCUD4g39Zv3UPZy07cM8A+/0nYKf4zP3A1UNb6dDIpGr4j81t9PY6ZWUDli8iUnQS\nfbO90GRSNXR197LzwLG4SxERyRsFyRBK19UA6PKWiJQUBckQmlofBIluuItIKVGQDKHxo4ZRXVWu\nLsAiUlIUJEPIzEjX1eiMRERKioJkiGXq1QVYREqLgmSIZepq2NFxjBOa6EpESoSCZIhlUjX09Do7\n2o/GXYqISF4oSIZYOhX23NLgjSJSIhQkQ2xqGCRb2xQkIlIaFCRDbFxNFWNGVOqMRERKhoIkAhq8\nUURKiYIkAplUDdt0aUtESoSCJAKZVA07DxzX/O0iUhIUJBFQzy0RKSUKkgj09dzSUCkiUgoUJBHo\nOyPR4I0iUgoUJBEYOayC+lHDdEYiIiVBQRIRdQEWkVKhIIlIpq6Gbfs03paIFD8FSUQy9TXsO9zJ\noeMn4i5FRCRSCpKI9M3f3qyzEhEpcgqSiPTN37513+GYKxERiZaCJCJTaqsx0xmJiBQ/BUlEhleW\nM2nMCLbpjEREipyCJELqAiwipUBBEqG+IHH3uEsREYmMgiRC6VQNB493036kK+5SREQioyCJ0FSN\nAiwiJUBBEqG05m8XkRKgIIlQ47gRVJSZzkhEpKgpSCJUWV7GlNpq9dwSkaKmIIlYOqXBG0WkuClI\nIpZJ1dCsLsAiUsQiDRIzu87MNpnZFjO7b4DtZmYPhNvXmdmsrG2PmtleM1vfb5/Pm1mrma0JXzdE\neQyDlU7VcOxED3sOdsZdiohIJCILEjMrB74GXA9MB243s+n9ml0PTAtfi4CHsrZ9G7juFB//VXef\nEb6WDWnhQ2xqSoM3ikhxi/KMZA6wxd23unsX8AQwv1+b+cDjHngBGGtmEwHc/edAe4T15UVfF2Dd\ncBeRYhVlkDQAO7Let4TrzrbNQD4ZXgp71MzGDa7MaE0cPZxhFWWav11EilYh3mx/CJgKzAB2AfcP\n1MjMFpnZSjNb2dbWls/6fkVZmZGu0+CNIlK8ogySVmBy1vvGcN3ZtvkV7r7H3XvcvRf4OsEltIHa\nPeLus919dn19/VkXP5Q0CrCIFLMog2QFMM3MMmZWBdwGLO3XZilwR9h7ax5wwN13ne5D++6hhBYA\n60/VNinSqRreaD9Kd09v3KWIiAy5yILE3buBe4FngY3Ak+6+wczuNrO7w2bLgK3AFoKzi9/v29/M\nvgv8ErjIzFrM7K5w05fN7GUzWwdcBXwqqmMYKlNTNZzocXa+eTzuUkREhlxFlB8eds1d1m/dw1nL\nDtxzin1vP8X63xnKGvMhkzV/+5S66pirEREZWoV4s73gpOvC4eR1n0REipCCJA9SI6sYNaxCN9xF\npCgpSPLAzILBG/dr8EYRKT4KkjwJugBrmBQRKT4KkjxJp2po7ThGZ3dP3KWIiAwpBUmeTE3V0Ouw\no12Xt0SkuChI8kTzt4tIsVKQ5Emmrwuw5m8XkSKjIMmTMdWV1NZUqQuwiBQdBUkeafBGESlGCpI8\n0nDyIlKMFCR5NLW+hj0HOznS2R13KSIiQ0ZBkkeZlG64i0jxUZDk0VuDN+pZEhEpHgqSPEqngiHk\nNVSKiBQTBUkeVVdVcN7o4WzTGYmIFBEFSZ5p8EYRKTYKkjxLp2po1nDyIlJEzhgkZlZuZomfF71Q\nTE3V0H6kizePdsVdiojIkDhjkLh7DzDg/Oly9voGb9SDiSJSLHK9tPW8mT1oZu81s1l9r0grK1J6\nlkREik1Fju1mhF+/mLXOgfcPbTnFb0ptNWUG2zScvIgUiZyCxN2virqQUlFVUUbjuGrN3y4iRSOn\nS1tmNsbMvmJmK8PX/WY2JuriilVaXYBFpIjkeo/kUeAQcGv4Ogh8K6qiit3UVA3N+47i7nGXIiIy\naLneI7nA3W/Jev8FM1sTRUGlIJOq4XBnN22HOxk/anjc5YiIDEquZyTHzOyKvjdm9h7gWDQlFb++\nLsAavFFEikGuZyR3A49n3RfpAO6MpqTiN/XksySHmZOpjbkaEZHBOWOQmFkZcJG7X25mowHc/WDk\nlRWxSWNHUFVepsEbRaQo5PJkey/wJ+HyQYXI4JWXGVPqqtVzS0SKQq73SH5iZp8xs8lmVtv3irSy\nIqf520WkWOR6j+S3w6/3ZK1zYOrQllM6ptbX8PPX2ujtdcrKLO5yRETOWa73SD7q7s/noZ6Ska6r\noau7l50HjtE4rjruckREzlmu90gezEMtJSWjLsAiUiRyvUfynJndYma6BjNEMlldgEVEClmuQfIJ\n4Emg08wOmtkhMztj7y0zu87MNpnZFjO7b4DtZmYPhNvXZQ9Nb2aPmtleM1vfb59aM1tuZq+FX8fl\neAyJMmH0MEZUlqsLsIgUvFyDZAzwMeB/ufto4FLgA6fbwczKga8B1wPTgdvNbHq/ZtcD08LXIuCh\nrG3fBq4b4KPvA55z92nAc+H7gmNmGrxRRIpCrkHyNWAeb82UeIgz3zeZA2xx963u3gU8Aczv12Y+\n8LgHXgDGmtlEAHf/OdA+wOfOBx4Llx8Dbs7xGBJnquZvF5EikGuQzHX3e4DjAO7eAVSdYZ8GYEfW\n+5Zw3dm26W+Cu+8Kl3cDE87QPrEyqRreaD/KiZ7euEsRETlnuQbJifBSlQOYWT0Q+08/D8ZhH3As\ndjNb1Dd/SltbW54ry006VUNPr9PSofEvRaRw5RokDwBLgPFm9r+B/wT+8gz7tAKTs943huvOtk1/\ne/ouf4Vf9w7UyN0fcffZ7j67vr7+DB8ZD/XcEpFikFOQuPt3CMbb+j/ALuBmd/+XM+y2AphmZhkz\nqwJuA5b2a7MUuCPsvTUPOJB12epUlvLWyMN3Ak/ncgxJ9FaQ6D6JiBSuXIdIwd1fBV49i/bdZnYv\n8CxQDjzq7hvM7O5w+8PAMuAGYAtwFPh43/5m9l3gSiBlZi3A/3T3bwJfAp40s7uA7QQzNhakcdWV\njBlRqTMSESloOQfJuXD3ZQRhkb3u4axl51fH78pud/sp1u8Hrh7CMmPzVhdgDd4oIoUr13skEpG+\n+dtFRAqVgiRm6boaWt88xvETPXGXIiJyThQkMcvUBzfct+vBRBEpUAqSmGXq1AVYRAqbgiRm6VQw\nF4m6AItIoVKQxGzU8EpSI4fpjERECpaCJAHUc0tECpmCJAEyqRq26lkSESlQCpIESKdq2He4k0PH\nT8RdiojIWVOQJIDmbxeRQqYgSYCTgzfu1+UtESk8CpIEOL+uGjPY1qYgEZHCoyBJgOGV5UwaM0Jd\ngEWkIClIEiKTqmGbhkkRkQKkIEmIdKqabW2HCUbWFxEpHAqShMikRnLweDcdR9UFWEQKi4IkITIn\nx9zSfRIRKSwKkoTIpEYCGrxRRAqPgiQhGseNoLzMdEYiIgVHQZIQleVlTKmt1tPtIlJwFCQJosEb\nRaQQKUgSJF1XQ/O+I+oCLCIFRUGSIJn6Go6d6GHPwc64SxERyZmCJEHemr9dl7dEpHAoSBIkU68g\nEZHCoyBJkImjhzOsokxdgEWkoChIEqSszEjX1eihRBEpKAqShEmnqnVGIiIFRUGSMJnUSN5oP0pP\nr7oAi0hhUJAkTCZVzYkep7XjWNyliIjkREGSMCcHb9T87SJSIBQkCZNJhV2A23SfREQKg4IkYVIj\nqxg5rIJmTbsrIgVCQZIwZqbBG0WkoChIEiidCgZvFBEpBJEGiZldZ2abzGyLmd03wHYzswfC7evM\nbNaZ9jWzz5tZq5mtCV83RHkMccikamjpOEpnd0/cpYiInFFkQWJm5cDXgOuB6cDtZja9X7PrgWnh\naxHwUI77ftXdZ4SvZVEdQ1wyqWp6HXa06z6JiCRflGckc4At7r7V3buAJ4D5/drMBx73wAvAWDOb\nmOO+RUvzt4tIIYkySBqAHVnvW8J1ubQ5076fDC+FPWpm44au5GR4azh5dQEWkeQrxJvtDwFTgRnA\nLuD+gRqZ2SIzW2lmK9va2vJZ36CNqa6ktqZKZyQiUhCiDJJWYHLW+8ZwXS5tTrmvu+9x9x537wW+\nTnAZ7Ne4+yPuPtvdZ9fX1w/qQOKQrtPgjSJSGKIMkhXANDPLmFkVcBuwtF+bpcAdYe+tecABd991\nun3Deyh9FgDrIzyG2GRSI2nWGYmIFICKqD7Y3bvN7F7gWaAceNTdN5jZ3eH2h4FlwA3AFuAo8PHT\n7Rt+9JfNbAbgQDPwiaiOIU6ZVDVPNR3naFc31VWR/TWJiAxapD+hwq65y/qtezhr2YF7ct03XP87\nQ1xmIvX13Gred5Tpk0bHXI2IyKkV4s32knBy8EY94S4iCacgSah0qhqAZg0nLyIJpyBJqOqqCs4b\nPZytbQoSEUk2BUmCpVPVOiMRkcRTkCRYJjVS90hEJPHUrzTBMqlq2o90ceDoCcZUV8Zdzhn19jqb\n9x7ixa3tvLStHQxuunwSV100nqoK/c4iUqwUJAmWPX/7jOqxMVfz67p7enll10Fe2tbOC1vbWdHc\nzoFjJwCYOGY4J3p6+dd1uxhXXcmHL5vEwlkNzJg8FjOLuXIRGUoKkgTLhD23tu07zIzJ8QdJV3cv\n61re5MVtwRnHqu0dHO7sBoIhXT546QTmZOqYm6mlcdwIenqdX7y2j8WrW3ly5Q7+8YXtTE3VsGBm\nAzfPbGBybXXMRyQiQ0FBkmCTa6sps/iGkz/W1cPqHR0nL1U1vdFBZ3cvANPGj+TmmZNOBseE0cN/\nbf+KcuOqi8dz1cXjOXj8BD96eTdPNbVw//LN3L98M3Mytdwyq4Hr3zGR0cOTf+lORAamIEmwYRXl\nNIwbkbcb7oc7u1nZHITGi9vaWdfyJid6HDOYPnE0/3XuFOZm6nhXehx1I4ed1WePHl7Jre+azK3v\nmsyO9qM8vaaVxU2t/OlTL/O5pzfwgekTWDirgfdOq6eyXPdTRAqJgiThgsEbowmSN492saK5gxe3\n7uel5nbWtx6g16GizHhH4xh+94oMczO1vPP8WsaMGLozhsm11dz7/mncc9WFrG05wOKmFn64difP\nrNtFamQVN14+iVtmNXLppNG6nyJSABQkCZepq6ZpewfuPugfqnsPHWfFtg5e2rafF7e18+ruQwBU\nVZQxY/JY7r3qQuZk6ph1/ti8DBRpZsyYPJYZk8fyFx+azs827WXJ6la+88IbfOv5ZqaNH8nCWY3c\nPHMSE8eMiLweETk3CpKEy6RqONzZzb7DXdSPOrvLSTvfPMaL2/afvFTV95R8dVU57zx/HB96x0Tm\nTq3jssYxDK8sj6L8nFVVlHHtpedx7aXnceDoCZ55eSeLm1r5qx+9ypeffZV3X1DHgpmNXPf28xg5\nTP9sRZJE/yMTLlPfN3/7kdMGibuzff/RoCtuGB4tHccAGDW8gjnpWn579mTmZGp5e8OYRN+HGFNd\nyUfmns9H5p5P874jLFndypLVrXzmX9byP36wnuvefh4LZjbwngtTlJfp0pdI3BQkCdc3f3vzviPM\nydSeXO/uvLb38MmuuC9t28+eg50A1NZUMSddy11XZJiTqeXi80YX7A/cdKqGT33gbfzRNdNYtb2D\nxatbeWbtTpasbmXC6GHMn9HAwlkNXHyehtoXiYuCJOEaxo2gstzY0naY9a0HwuDYz4rmDtqPdAEw\nYfQw5mbqmJOpZd7UWi6oH1l0N6nNjNnpWmana/nch6fz01f38lRTK4/+5zYe+flWLpk4mltmNXDT\njEmMH/XrXZFFJDoWzC1V3GbPnu0rV66Mu4xzdvX9P+P1rFGAJ9eOOBkcczO1TKmtLrrgyNX+w508\ns24Xi5taWNtygDKD906rZ+GsBq6dfh4jquK99yNSyMxslbvPPmM7BUnyfX9VC01vdDA3U8u70rVM\nGqseTAPZsvcwS1a38IPVO2l98xg1VeVc/46JLJzVwLxMHWUFenlPJC4KkiyFHiRydnp7nRe3tbNk\ndQvLXt7N4c5uJo0Zzs0zg/spF44fFXeJIgVBQZJFQVK6jnX1sHzjHhY3tfCL1/bR0+tc1jiGBTMb\nuOnySWf9hL5IKVGQZFGQCAQPZC5dE/T42rDzIBVlxpUX1bNgZiNXXzI+9mdpRJJGQZJFQSL9bdp9\niMWrW/jB6lb2HOxk1PAKPnzZRBbOamT2+eNKtvOCSDYFSRYFiZxKT6/zy9f3s7iphR9t2M3Rrh4m\n145gwcxGFs5sIJ2qibtEkdgoSLIoSCQXRzq7eXbDbhY3tfL86/twh1lTxrJgViM3XjaRsdVVcZco\nklcKkiwKEjlbuw4c4+k1O1nc1MLmPYepKi/j/RePZ8GsBk0dLCVDQZJFQSLnyt3ZsPMgS1a38vSa\nVvYd7mJsdSU3XjaJBbMamKmpg6WIKUiyKEhkKHT39PKLLftY3NTKjzfsprO7l0yqhoWaOliKlIIk\ni4JEhtqh4yf4t5d3s3h1Cy9sbQdgTqaWhTMbuOEyTR0sxUFBkkVBIlFq6TjK02t28lRTC1vbjjCs\nooxrpk/gFk0dLAVOQZJFQSL54O6sC6cOXrp2Jx1HT1BXU8VNMzR1sBQmBUkWBYnkW1d3L/+xuY3F\nTS08t3EvXT29mjpYCo6CJIuCROLUN3XwkqZWVm7vwAxNHSwFQUGSRUEiSbF9fzB18OKmVt5oP8qI\nynI+eOkEFs5q1NTBkjgKkiwKEkkad6fpjQ6eagqmDj54vJvxo4Zx88wGFsxs4JKJmjpY4qcgyaIg\nkSTr7O7h3zfuZfHqVn766l66e51LJo5m4cwG5s+YxPjRmjpY4pGIIDGz64C/BcqBb7j7l/ptt3D7\nDcBR4GPu3nS6fc2sFvgekAaagVvdveN0dShIpFC0H+nimXU7eaqplbU73qTM4Ipp9dyiqYMlBrEH\niZmVA5uBDwAtwArgdnd/JavNDcAnCYJkLvC37j73dPua2ZeBdnf/kpndB4xz9z89XS0KEilEr7cd\nZklTK0tWt2rqYIlFrkESZXeROcAWd98aFvQEMB94JavNfOBxD9LsBTMba2YTCc42TrXvfODKcP/H\ngJ8Bpw0SkUJ0Qf1IPvPBi/j0B97GS83tLGlq5V9f3sX3V7Vo6mBJlCiDpAHYkfW+heCs40xtGs6w\n7wR33xUu7wYmDFXBIklUVmbMm1rHvKl1fGH+pSx/JZg6+B9+vpW//9nrTKmtZphGI5ZT+MuF7+Bd\n6dpIv0dBd2B3dzezAa/NmdkiYBHAlClT8lqXSFSGV5Zz4+WTuPHySbQd6mTp2p00be/AKf5OM3Ju\nRuRhCukog6QVmJz1vjFcl0ubytPsu8fMJrr7rvAy2N6Bvrm7PwI8AsE9knM9CJGkqh81jLuuyHDX\nFZm4S5ESF+X58ApgmpllzKwKuA1Y2q/NUuAOC8wDDoSXrU6371LgznD5TuDpCI9BRETOILIzEnfv\nNrN7gWcJuvA+6u4bzOzucPvDwDKCHltbCLr/fvx0+4Yf/SXgSTO7C9gO3BrVMYiIyJnpgUQRERlQ\nrt1/1dVDREQGRUEiIiKDoiAREZFBUZCIiMigKEhERGRQSqLXlpm1EXQV7pMC9sVUzrlSzfmhmvND\nNefPYOo+393rz9SoJIKkPzNbmUuXtiRRzfmhmvNDNedPPurWpS0RERkUBYmIiAxKqQbJI3EXcA5U\nc36o5vxQzfkTed0leY9ERESGTqmekYiIyBApqSAxs+vMbJOZbQnne08EM5tsZj81s1fMbIOZ/WG4\nvtbMlpvZa+HXcVn7/Fl4HJvM7IMx1l5uZqvN7JlCqDmczvn7ZvaqmW00s98ogJo/Ff67WG9m3zWz\n4Ums2cweNbO9ZrY+a91Z12lm7zSzl8NtD5hZZJPTn6Lmvw7/fawzsyVmNjbpNWdt+2MzczNL5bVm\ndy+JF8Fw9K8DU4EqYC0wPe66wtomArPC5VHAZmA68GXgvnD9fcBfhcvTw/qHAZnwuMpjqv3TwD8D\nz4TvE10z8Bjwe+FyFTA2yTUTTDu9DRgRvn8S+FgSawbeB8wC1metO+s6gZeAeYAB/wZcn+earwUq\nwuW/KoSaw/WTCabe2A6k8llzKZ2RzAG2uPtWd+8CngDmx1wTAO6+y92bwuVDwEaCHyDzCX7wEX69\nOVyeDzzh7p3uvo1gPpc5+a0azKwR+BDwjazVia3ZzMYQ/Cf8JoC7d7n7m0muOVQBjDCzCqAa2EkC\na3b3nwPt/VafVZ0WzHo62t1f8OCn3eNZ++SlZnf/sbt3h29fIJihNdE1h74K/An8yrzLeam5lIKk\nAdiR9b4lXJcoZpYGZgIvAhM8mDESYDcwIVxOyrH8DcE/3N6sdUmuOQO0Ad8KL8d9w8xqSHDN7t4K\n/F/gDWAXwSyiPybBNfdztnU2hMv918fldwl+W4cE12xm84FWd1/bb1Neai6lIEk8MxsJPAX8kbsf\nzN4W/taQmC52ZvZhYK+7rzpVm6TVTPCb/SzgIXefCRwhuNxyUtJqDu8pzCcIwUlAjZl9NLtN0mo+\nlUKps4+ZfRboBr4Tdy2nY2bVwJ8Dn4urhlIKklaCa4h9GsN1iWBmlQQh8h13Xxyu3hOeghJ+3Ruu\nT8KxvAe4ycyaCS4Tvt/M/olk19wCtLj7i+H77xMES5JrvgbY5u5t7n4CWAy8m2TXnO1s62zlrUtJ\n2evzysw+BnwY+EgYgJDcmi8g+EVjbfj/sRFoMrPzyFPNpRQkK4BpZpYxsyrgNmBpzDUBEPaW+Caw\n0d2/krVpKXBnuHwn8HTW+tvMbJiZZYBpBDfO8sbd/8zdG909TfBn+e/u/tGE17wb2GFmF4WrrgZe\nIcE1E1zSmmdm1eG/k6sJ7qElueZsZ1VneBnsoJnNC4/3jqx98sLMriO4ZHuTux/N2pTImt39ZXcf\n7+7p8P9jC0Hnnd15qzmqngVJfAE3EPSIeh34bNz1ZNV1BcEp/zpgTfi6AagDngNeA34C1Gbt89nw\nODYRYQ+RHOu/krd6bSW6ZmAGsDL8s/4BMK4Aav4C8CqwHvhHgh44iasZ+C7BfZwTBD/M7jqXOoHZ\n4bG+DjxI+OB0HmveQnBfoe//4sNJr7nf9mbCXlv5qllPtouIyKCU0qUtERGJgIJEREQGRUEiIiKD\noiAREZFBUZCIiMigKEik6Fgwwu/vn+O+y7JHez1Fmy+a2TXnVt1pP/djZvbgGdpcaWbvHqLv9/+G\n4nNEFCRSjMYCAwZJOPDhKbn7DR4M5Hi6Np9z958Mor7BuJLgyfZBc/ch+RwRBYkUoy8BF5jZmnBu\niSvN7BdmtpTgSXbM7AdmtsqCeT4W9e1oZs1mljKztAXzlXw9bPNjMxsRtvm2mf1WVvsvmFlTOLfD\nxeH6egtCC1gDAAAC7klEQVTm39gQDg65PXuOiKzv93Ez22xmLxEMO9O3/kYzezEcXPInZjYhHNDz\nbuBT4bG9d6B2A3yPS83spXCfdWY2LVx/OPz6xXDbGjNrNbNvhes/mrXfP5hZ+ZD87UjxyddTr3rp\nla8XkOZX55e4kmCAxkzWutrw6wiCp3vrwvfNQCr8jG5gRrj+SeCj4fK3gd/Kav/JcPn3gW+Eyw8C\nfxYuX0cwckGqX50TCYZAqSeYG+V54MFw2zjemgr794D7w+XPA5/J+owB2/X7Pn9HMGYU4ffpm9vk\ncL92Y4GXgXcClwA/BCrDbX8P3BH3361eyXyd9jRfpIi85MF8DH3+wMwWhMuTCcYg2t9vn23uviZc\nXkUQLgNZnNVmYbh8BbAAwN1/ZGYdA+w3F/iZu7cBmNn3gLeF2xqB74UDHVYRTG41kFza/RL4rAXz\nxyx299f6NwjHW/on4CvuvsrM7iUIlBXBJkbw1oCLIr9Cl7akVBzpWzCzKwlG1f0Nd78cWA0MH2Cf\nzqzlHjjlL16dObQ5W39HcHbyDuATp6gvp3bu/s/ATcAxYJmZvX+Az/k8wcjI3wrfG/CYu88IXxe5\n++cHc0BSvBQkUowOEUxZfCpjgA53Pxre05gXQQ3PA7cCmNm1BJeg+nsR+E0zq7NgGoH/0q/GvmG9\n78xa3//YTtXuJDObCmx19wcIRni9rN/2GwmC9Q+yVj8H/JaZjQ/b1JrZ+QMfqpQ6BYkUHXffDzxv\nZuvN7K8HaPIjoMLMNhLcmH8hgjK+AFxrZusJAmI3QQhk17mL4EzglwTBszFr8+eBfzGzVcC+rPU/\nBBb03Ww/TbtstwLrzWwN8HaCaVWzfZpgdry+G+tfdPdXgL8Afmxm64DlBPd0RH6NRv8ViYCZDQN6\n3L3bzH6DYFbGGXHXJRIF3WwXicYU4EkzKwO6gP8Wcz0ikdEZiYiIDIrukYiIyKAoSEREZFAUJCIi\nMigKEhERGRQFiYiIDIqCREREBuX/A+Ws7zECEtbiAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x118ee9cf8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(svm_sizes, svm_errs)\n",
    "plt.xlabel('training set size')\n",
    "plt.ylabel('error')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# (e)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x1132c91d0>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuYXmV97//3Z045Tk4wyYxJYAYMCRElhBECJBZRasBD\nwCqFnxqwtjEFbGvbvUtrdzf6u/z9qC3aUhAERKEqBxUk251dBMRLRQJMEENCCAxJIIk5DJDzJJnT\nd/+x1pAnkzk8k5lnnmdmPq/rWtez1r3u9cx3QTLf3Ou+130rIjAzMztWRfkOwMzMBjcnEjMz6xMn\nEjMz6xMnEjMz6xMnEjMz6xMnEjMz6xMnEjMz6xMnEjMz6xMnEjMz65OSXH65pIXAvwPFwJ0RcUOH\n80rPXww0AldFxHMZ54uBOmBLRHwkLZsE3A9UAxuByyJiZ3dxHH/88VFdXd0/N2VmNkysXLnyjYio\n6KlezhJJmgRuAS4ENgPPSloWES9mVLsImJFuZwO3pp/t/hJYC4zLKLsOeDwibpB0XXr8d93FUl1d\nTV1dXR/vyMxseJH0Wjb1cvlo6yygPiLWR0QTcB+wqEOdRcA9kVgBTJBUBSBpGvBh4M5Orrk73b8b\nuCRXN2BmZj3LZSKZCmzKON6clmVb59+A/w60dbhmSkRsTfe3AVP6JVozMzsmBdnZLukjwI6IWNld\nvUimLu50+mJJSyTVSapraGjIRZhmZkZuE8kWYHrG8bS0LJs65wEfk7SR5JHYBZK+l9bZnvH4qwrY\n0dkPj4jbI6I2ImorKnrsKzIzs2OUy0TyLDBDUo2kMuByYFmHOsuAxUrMA3ZHxNaI+PuImBYR1el1\nP4+IT2dcc2W6fyXwcA7vwczMepCzUVsR0SLpWuARkuG/d0XEGklL0/O3ActJhv7Wkwz//WwWX30D\n8ICkzwGvAZflIn4zM8uOhsMKibW1teHhv2ZmvSNpZUTU9lSvIDvbC8UTL+3gm7+oz3cYZmYFzYmk\nG0/Wv8G/P/YKrW1Dv9VmZnasnEi6MbOynEMtbWx8c3++QzEzK1hOJN04tSqZmWXdtr15jsTMrHA5\nkXTjnZPHUiR4aeuefIdiZlawnEi6MbK0mOrjx/CSWyRmZl1yIunBrMpy1m13IjEz64oTSQ9mVY7j\n9bca2X+oJd+hmJkVJCeSHsysLCcCXnarxMysU04kPZhVWQ545JaZWVecSHowfeJoRpcVu8PdzKwL\nTiQ9KCoSp0wpd4vEzKwLTiRZmFVZzkvb9jAcJrg0M+stJ5IszKwsZ2djMw17D+U7FDOzguNEkoWZ\naYe7+0nMzI7mRJKFWZWec8vMrCtOJFmYNKaMyeUjWLvNc26ZmXWU00QiaaGkdZLqJV3XyXlJuik9\nv0rS3LR8pKRnJP1O0hpJX8645npJWyQ9n24X5/Ie2s2s9MgtM7PO5CyRSCoGbgEuAmYDV0ia3aHa\nRcCMdFsC3JqWHwIuiIjTgTnAQknzMq77RkTMSbflubqHTLMqy3llxz5aWtsG4seZmQ0auWyRnAXU\nR8T6iGgC7gMWdaizCLgnEiuACZKq0uN9aZ3SdMvr2NtZleNoamlj45uN+QzDzKzg5DKRTAU2ZRxv\nTsuyqiOpWNLzwA7g0Yh4OqPeF9JHYXdJmtjZD5e0RFKdpLqGhoa+3kvGyC33k5iZZSrYzvaIaI2I\nOcA04CxJp6WnbgVOInnktRW4sYvrb4+I2oioraio6HM875w8luIiuZ/EzKyDXCaSLcD0jONpaVmv\n6kTELuAJYGF6vD1NMm3AHSSP0HJuZGkxNV7kyszsKLlMJM8CMyTVSCoDLgeWdaizDFicjt6aB+yO\niK2SKiRNAJA0CrgQeCk9rsq4/lJgdQ7v4QgeuWVmdrSSXH1xRLRIuhZ4BCgG7oqINZKWpudvA5YD\nFwP1QCPw2fTyKuDudORXEfBARPw0Pfc1SXNIOt83Ap/P1T10NGtKOf971Vb2HWph7Iic/aczMxtU\ncvrbMB2au7xD2W0Z+wFc08l1q4AzuvjOz/RzmFlr73B/efte5p7QaR+/mdmwU7Cd7YXo1CpPlWJm\n1pETSS9MnTCKMWXFTiRmZhmcSHqhqEicUlnO2q1+l8TMrJ0TSS/Nqixn3fa9XuTKzCzlRNJLsyrH\nsauxmR1e5MrMDHAi6TUvcmVmdiQnkl6a1Z5I3E9iZgY4kfTahNFlTBk3wiO3zMxSTiTHYFblOD/a\nMjNLOZEcg1mV5dR7kSszM8CJ5JjMrCynqbWNDW/sz3coZmZ550RyDDxyy8zsMCeSY+BFrszMDnMi\nOQYjSoo5yYtcmZkBTiTHbGZluddvNzPDieSYzaosZ/POA+w71JLvUMzM8iqniUTSQknrJNVLuq6T\n85J0U3p+laS5aflISc9I+p2kNZK+nHHNJEmPSnol/czLClOzKr02iZkZ5DCRpMvk3gJcBMwGrpA0\nu0O1i4AZ6bYEuDUtPwRcEBGnA3OAhema7gDXAY9HxAzg8fR4wLWP3HIiMbPhLpctkrOA+ohYHxFN\nwH3Aog51FgH3RGIFMEFSVXq8L61Tmm6Rcc3d6f7dwCU5vIcuTZs4irEjStxPYmbDXi4TyVRgU8bx\n5rQsqzqSiiU9D+wAHo2Ip9M6UyJia7q/DZjS34FnQxKnTBnrkVtmNuwVbGd7RLRGxBxgGnCWpNM6\nqRMcbqkcQdISSXWS6hoaGnIS46yqcazb5kWuzGx4y2Ui2QJMzzielpb1qk5E7AKeABamRdslVQGk\nnzs6++ERcXtE1EZEbUVFxTHfRHdmVZaz+0Az2/YczMn3m5kNBrlMJM8CMyTVSCoDLgeWdaizDFic\njt6aB+yOiK2SKiRNAJA0CrgQeCnjmivT/SuBh3N4D92aOcVTpZiZ5SyRREQLcC3wCLAWeCAi1kha\nKmlpWm05sB6oB+4Ark7Lq4AnJK0iSUiPRsRP03M3ABdKegX4YHqcFx4CbGYGJbn88ohYTpIsMstu\ny9gP4JpOrlsFnNHFd74JfKB/Iz0240eXUjV+pBOJmQ1rBdvZPljMrCxnrZfdNbNhzImkj2ZWlvNq\nwz6avciVmQ1TTiR9NKuynObW8CJXZjZsOZH0UXuHu0dumdlw5UTSRydXjKWkSLzkfhIzG6acSPqo\nrKSIkyrGeOSWmQ1bTiT9YGblOD/aMrNhy4mkH8yqLGfLrgPsOdic71DMzAacE0k/mJWuTfKyWyVm\nNgw5kfSD9kWu/HjLzIYjJ5J+MHXCKMpHlLjD3cyGJSeSfiCJmZXlTiRmNiw5kfSTmZXlrN22x4tc\nmdmwk9PZf4eTWZXlfP/pFv74WysoLlK/f/85Jx/Hte9/J0U5+G4zs75wIukn7581mQUvbudQcxut\nbf3bKjnQ3MrXH32Z1Vt282+Xz2F0mf+3mVnh0HB4FFNbWxt1dXX5DqNPvvvkBr7y0xeZ/Y5x3Ln4\nvVSOH5nvkMxsiJO0MiJqe6rnPpJB4qrzarjzylo2NOxn0S2/ZvWW3fkOycwMyHEikbRQ0jpJ9ZKu\n6+S8JN2Unl8laW5aPl3SE5JelLRG0l9mXHO9pC2Snk+3i3N5D4XkgllT+NGfn0uxxCdve4pHX9ye\n75DMzHKXSCQVA7cAFwGzgSskze5Q7SJgRrotAW5Ny1uAv4mI2cA84JoO134jIuak2xFL+Q51p1aN\n4yfXnscpU8ay5D/ruOOX6z1SzMzyKpctkrOA+ohYHxFNwH3Aog51FgH3RGIFMEFSVURsjYjnACJi\nL7AWmJrDWAeVyeUjuW/JOVx0WiVfXb6Wf3joBa/QaGZ5k8tEMhXYlHG8maOTQY91JFUDZwBPZxR/\nIX0UdpekiZ39cElLJNVJqmtoaDi2Oyhgo8qKufmKuVzz/pO595lNXPWdZ9jd6EkjzWzgFXRnu6Sx\nwI+Bv4qI9pWjbgVOAuYAW4EbO7s2Im6PiNqIqK2oqBiQeAdaUZH4bx+axb9+8nSe2fAWH7/1SV57\n00v+mtnAymUi2QJMzzielpZlVUdSKUkS+X5EPNheISK2R0RrRLQBd5A8QhvWPnHmNL73ubN5c38T\nl9zyJM9ufCvfIZnZMJLLRPIsMENSjaQy4HJgWYc6y4DF6eitecDuiNgqScC3gbUR8fXMCyRVZRxe\nCqzO3S0MHmefdBwPXX0eE0eX8ak7nuah327Od0hmNkzkLJFERAtwLfAISWf5AxGxRtJSSUvTasuB\n9UA9Sevi6rT8POAzwAWdDPP9mqQXJK0C3g98MVf3MNjUHD+GB68+lzNPnMgX7/8dN/5sHW39/Ja9\nmVlHfrN9CGpqaeN//GQ199dt4sPvqeLGT57OyNLifIdlZoNMtm+2e9KmIaispIgb/ujd1FSM4Z//\n6yW27DzAHYtrqSgfke/QzGwIKuhRW3bsJLH0D07m1k+dyUvb9nDJLU96vRQzywknkiFu4WmV/PDz\n59Lc2sYf3fobfrFuR75DMrMhxolkGHj3tPE8fO15nDBpNH/y3We556mN+Q7JzIYQJ5Jhomr8KH64\n9BwumDWZf3p4DdcvW0OLp1Uxs37gRDKMjBlRwrc+U8ufzq/hu7/ZyJ/eU8feg55Wxcz6xomkOy//\nDP7P38EQGiJdXCT+8SOz+f8ufTe/euUNPnnbU2zZdSDfYZnZIOZE0p3Nz8DTt8Ev/yXfkfS7/+fs\nE/juZ9/Lll0HWHTzkzy/aVe+QzKzQcqJpDvv/xKcfgU88VX47ffyHU2/WzCjgoeuPpdRZUX88bee\n4n+v2prvkMxsEHIi6Y4EH70JTjoflv0F1D+W74j63Tsnl/OTq8/j3VPHc80PnuOWJ+q9UJaZ9YoT\nSU9KyuCy/4TJs+GBK+H3z+c7on533NgRfP/PzuaSOe/gXx5Zx9/+cBWHWlrzHZaZDRJOJNkYOQ4+\n9UMYOQF+cBnsfC3fEfW7ESXFfOOP5/DXF57Cj5/bzGfufIa39jflOywzGwScSLI1rgo+/SNoOQjf\n/wQ0Dr01PyTxFx+YwU1XnMHzm3dx6Tef5NWGffkOy8wKXI+JRFKxJE/VDjD5VLj8B7BzI9x7BTQf\nzHdEOfGx09/BvX82j30HW7j0lif5Tf0b+Q7JzApYj4kkIlqBKwYglsGhej5c+i3YtAIe/DNoG5pv\nh5954kR+cs15VI4fyeK7nuH+Z1/Pd0hmVqCyfbT1pKSbJS2QNLd9y2lkhey0j8MffhXWLoOffSnf\n0eTM9Emj+dGfn8u57zyev/vxC/z/y9d6oSwzO0q2iWQO8C7gK8CN6favPV0kaaGkdZLqJV3XyXlJ\nuik9v6o9OUmaLukJSS9KWiPpLzOumSTpUUmvpJ8Ts7yH/nXONXD2n8OKb8Jvbs5LCANh3MhS7rqy\nlsXnnMi3frmepd9bSWNTS77DMrMCklUiiYj3d7Jd0N01koqBW4CLgNnAFZJmd6h2ETAj3ZYAt6bl\nLcDfRMRsYB5wTca11wGPR8QM4PH0eOBJ8KGvwqkfS1olq3+clzAGQklxEV9ZdBrXf3Q2j63dzmXf\neorte4Zm/5CZ9V5WiUTSeElfl1SXbjdKGt/DZWcB9RGxPiKagPuARR3qLALuicQKYIKkqojYGhHP\nAUTEXpI136dmXHN3un83cEk295ATRcXw8TvghHPgoaWw8cm8hTIQrjqvhjuvrGVDw34W3fwkq7fs\nzndIZlYAsn20dRewF7gs3fYA3+nhmqnApozjzRxOBlnXkVQNnAE8nRZNiYj2uTy2AVOyuYGcKR2Z\njOSaWA33XQH7hvbCURfMmsKP/vxcigSfvO0pHn1xe75DMrM8yzaRnBwR/zNtXayPiC8DJ+UyMABJ\nY4EfA38VEXs6no9kLo9Oe38lLWlvQTU0NOQ20NGTkpFcB3dD/eO5/VkF4NSqcfzk2vM4pbKcJf9Z\nxx2/XO9pVcyGsWwTyQFJ89sPJJ0H9DT3+BZgesbxtLQsqzqSSkmSyPcj4sGMOtslVaV1qoBOmwAR\ncXtE1EZEbUVFRQ+h9oOqOTBqImz8Ve5/VgGYXD6S+5fM4+LTqvjq8rX8w0OrafZCWWbDUraJZClw\ni6SNkjYCNwOf7+GaZ4EZkmoklQGXA8s61FkGLE5Hb80DdkfEVkkCvg2sjYivd3LNlen+lcDDWd5D\nbhUVwYnnDZtEAjCytJj/uOIMrnn/ydz7zOtc9Z1n2H3AC2WZDTfZvNleBMyMiNOB9wDviYgzImJV\nd9dFRAtwLfAISWf5AxGxRtJSSUvTasuB9UA9cAdwdVp+HvAZ4AJJz6fbxem5G4ALJb0CfDA9Lgw1\n74Ndrw/Jubi6UlQk/tuHZvGvnzydZza8xce/+SSvvbk/32GZ2QBSNs+2JdVFRO0AxJMTtbW1UVdX\nl/sftP1FuPUcWHQLnPHp3P+8AvP0+jf5/PdWIuD2xbW8t3pSvkMysz6QtDKb3/3ZPtp6TNLfpi8K\nTmrf+hjj0FMxC0YfBxt/ne9I8uLsk47joavPY+LoMj51x9N8/WfreODZTTyyZhtPr3+Tddv2sn3P\nQQ42e4p6s6GkJMt6f5x+XpNRFgzAyK1BpagomYtrw6+Sdd6lfEc04GqOH8ODV5/LF+79LTf9vL7L\neiNLi5gwqowJo0uTLd0fn7E/YdSRxxNHlzGytAgNw/+uZoWsx0SS9pF8OiKG9tt2/aV6Abz4MOzc\nAJOGZ56dMLqMe/7kLPY3tbL7QDO7GpvY1dicbAeS/SPKDzSz4Y397DrQxM7GZppauh79VVZSxIRR\nh5PP+DThJAmpjPHp/sSM/QmjyxhTVuwEZJYjPSaSiGiTdDPJS4HWk+oFyefGXw/bRALJ2iZjR5Qw\ndkQJUyeMyvq6iOBgc9vbCSdJOocTTlLW9HZS2vRWI6vT8gPdPDIrKVLS4hmVJJb21s7EdD9pDZUd\nlaTKR5RQVOQEZNadbB9tPS7pj4AHw2+eda9iJoyZnDzemrs439EMOpIYVVbMqLJRVI3PPgEBHGxu\nZc+BJOHs3N/ErgPN7M5oBWUeb9tzkJe27WVXYxP7m7pOQEXicPJ5u/WT0dppP87YnzCqlHGjSil2\nArJhIttE8nngi0CrpIOASF4sH5ezyAYrKekn2Th8+0nyZWRpMSNLi5k8bmSvrmtqaWP3gYyWz9ut\nnyZ2H2hmZ+Phx3Fv7GuivmEfuxqb2Xuw61mQpWTm5MN9PZmtnQ7HGUlq/KhSSoq9cKkNLtkmkvHA\np4CaiPiKpBOAqtyFNchVz4c1D8Jb6+G4k/MdjfWgrKSIivIRVJSP6NV1La1t7DnYkjxqO6ovqJnd\nb5cnx6+/uZ+djc3sOdhMd+368hElSQunk36gw30/h5PPuFGljBlRwujSYj+Gs7zINpHcArQBF5Cs\nSbKXZPqS9+YorsGt5n3J54ZfOpEMYSXFRUwaU8akMWW9uq61Ldh78OiWT8cBCe0J6ve7Drxdr6d1\nxUaVFjNmRDGjy0oYXVacJJiyYsaUlTBmRMnb58aUFTN6RIfPsozzGfXcQrKeZJtIzo6IuZJ+CxAR\nO9NpT6wzx70TxlYmHe61n813NFZgiouUtih691eorS3Y19SS9PM0po/cDiSP3A40tbD/UCuNTS3s\nb2ql8VD62dTC3oMtbNt9kMamVvY3tdB4qJWmXsyLNqKk6IiENHpE+pmZqDqcHzuipNuEVVbi5DSU\nZJtImtOFqgJAUgVJC8U6434Sy4GiIjFuZCnjRpYyvY+vAze1tHGgPbGkSag9ySRlrew/dPizs3Nv\n7Dt0RPnB5ux/JZQW6+hEcwytpsyENqLE7xjlS7aJ5CbgIWCypK8CnwD+MWdRDQU1C2D1j+CNV6Di\nlHxHY3aEspIiykqKGD+6tN++s7UtaOwiCb3dWuqi1dRe/vtdB446n63iInXfasosH5HxuK+bRDWq\n1O8fZSOrRBIR35e0EvgAyYitSyJibU4jG+zefp/kV04kNiwUF4nykaWUj+y/5NTWFhxo7r61lPnI\nrrN6O/YePOpcT31N7SQYXXpkohk7IrvHex3PJ9cNzUER2bZIiIiXgJdyGMvQMukkKH9Hkkje+7l8\nR2M2KBUVKX3cVQLl/fOdEcGhlrYjklCXLaZOWk37D7Wyc38Tm3ceeLvVtP9QCy3ZZie6HxTR21ZT\nIQyKyDqRWC9JyeOtV3/ufhKzAiLp7XeOjuvH721qaTvqsdz+NBkd1WrqpFW171ALO/YcOuL8oW6m\nC+qorKToiESTJKAS/vZDM5kzfUI/3unRnEhyqXo+rLofGtbB5Fn5jsbMcijpdypjwuj++87m1jYa\nO2kRHdXP1FnLKU1GA8GJJJcy+0mcSMysl0qLixg/qojxo/qv3ykXPJg7lyZWw/jpyYuJZmZDVE4T\niaSFktZJqpd0XSfnJemm9PwqSXMzzt0laYek1R2uuV7Slk6W4C087e+TvPYktPm1GzMbmnKWSNIX\nGG8BLgJmA1dImt2h2kXAjHRbAtyace67wMIuvv4bETEn3Zb3a+D9rXoBNL4JDR4tbWZDUy5bJGcB\n9RGxPiKagPuARR3qLALuicQKYIKkKoCI+CXwVg7jGxjV85PPDb/KbxxmZjmSy0QyFdiUcbw5Lett\nnc58IX0UdpekiZ1VkLREUp2kuoaGht7E3b8mnggTTkg63M3MhqDB2Nl+K8la8XOArcCNnVWKiNsj\nojYiaisqKgYyvqNVv8/9JGY2ZOUykWwBpmccT0vLelvnCBGxPSJaI6INuIPkEVphq1kAB3bC9tU9\n1zUzG2RymUieBWZIqkmnnL8cWNahzjJgcTp6ax6wOyK2dvel7X0oqUuBwv/t3N5PsvHX+Y3DzCwH\ncpZIIqIFuBZ4BFgLPBARayQtlbQ0rbYcWA/Uk7Qurm6/XtK9wFPATEmbJbVPWPU1SS9IWgW8n2QJ\n4MI2fhpMrHE/iZkNSTl9sz0dmru8Q9ltGfsBXNPFtVd0Uf6Z/oxxwNQsgBcfhrZWKCrOdzRmZv1m\nMHa2D07VC+Dgbtj2Qr4jMTPrV04kAyVz3i0zsyHEiWSgjKtK1nJ3h7uZDTFOJAOpej689htoHZip\nnc3MBoITyUCqXgCH9sC23+U7EjOzfuNEMpDe7ifx4y0zGzqcSAZS+RQ4/hRP4GhmQ4oTyUCrXgCv\nPwWtzfmOxMysXziRDLTq+dC0D7a6n8TMhgYnkoHW3k/i5XfNbIhwIhloYyug4lS/mGhmQ4YTST5U\nz4fXV7ifxMyGBCeSfKhZAM2NsOW5fEdiZtZnTiT5cGL7+iTuJzGzwc+JJB/GHAeT3+UXE81sSHAi\nyZeaBfD609ByKN+RmJn1SU4TiaSFktZJqpd0XSfnJemm9PwqSXMzzt0laYek1R2umSTpUUmvpJ8T\nc3kPOVO9AFoOuJ/EzAa9nCUSScXALcBFwGzgCkmzO1S7CJiRbkuAWzPOfRdY2MlXXwc8HhEzgMfT\n48HnxHMBeRiwmQ16uWyRnAXUR8T6iGgC7gMWdaizCLgnEiuACZKqACLil8BbnXzvIuDudP9u4JKc\nRJ9roydB5Wl+MdHMBr1cJpKpwKaM481pWW/rdDQlIram+9uAKX0JMq+q3webn4Xmg/mOxMzsmA3q\nzvaICCA6OydpiaQ6SXUNDQ0DHFmWqudDy0HYUpfvSMzMjlkuE8kWYHrG8bS0rLd1Otre/vgr/dzR\nWaWIuD0iaiOitqKioleBD5gTzwUVeVp5MxvUcplIngVmSKqRVAZcDizrUGcZsDgdvTUP2J3x2Kor\ny4Ar0/0rgYf7M+gBNWoCVL7H75OY2aCWs0QSES3AtcAjwFrggYhYI2mppKVpteXAeqAeuAO4uv16\nSfcCTwEzJW2W9Ln01A3AhZJeAT6YHg9e1fNh8zPQfCDfkZiZHRMl3QxDW21tbdTVFWg/xMuPwA8u\ng8XL4KQ/yHc0ZmZvk7QyImp7qjeoO9uHhBPOSfpJ/HjLzAYpJ5J8GzkOqub4xUQzG7ScSApBzQLY\nXAdNjfmOxMys15xICkH1Amhrhk1P5zsSM7NecyIpBCfMAxX78ZaZDUpOJIVgRDlMnesOdzMblJxI\nCkX1fNiyEg7ty3ckZma94kRSKKoXQFsLbFqR70jMzHrFiaRQnDAPikr8eMvMBh0nkkJRNgamnukJ\nHM1s0HEiKSTVC+D3v4VDe/MdiZlZ1pxICknNAohWeN39JGY2eDiRFJJpZ0FRqZffNbNBxYmkkJSN\nhmnv9YuJZjaoOJEUmpoFsPV3cHB3viMxM8uKE0mhqZ4P0QavPZXvSMzMsuJEUmimnQXFI/x4y8wG\njZwmEkkLJa2TVC/puk7OS9JN6flVkub2dK2k6yVtkfR8ul2cy3sYcKUjYfpZTiRmNmjkLJFIKgZu\nAS4CZgNXSJrdodpFwIx0WwLcmuW134iIOem2PFf3kDfV82HrKjiwM9+RmJn1KJctkrOA+ohYHxFN\nwH3Aog51FgH3RGIFMEFSVZbXDl3VC4CA136T70jMzHqUy0QyFdiUcbw5LcumTk/XfiF9FHaXpImd\n/XBJSyTVSapraGg41nvIj2m1UDLS826Z2aAwGDvbbwVOAuYAW4EbO6sUEbdHRG1E1FZUVAxkfH1X\nMiLpJ/G8W2Y2COQykWwBpmccT0vLsqnT5bURsT0iWiOiDbiD5DHY0FP9Pti+GhrfynckZmbdymUi\neRaYIalGUhlwObCsQ51lwOJ09NY8YHdEbO3u2rQPpd2lwOoc3kP+1LT3kzyZ70jMzLpVkqsvjogW\nSdcCjwDFwF0RsUbS0vT8bcBy4GKgHmgEPtvdtelXf03SHCCAjcDnc3UPefWOuVA6Onm8depH8x2N\nmVmXFBH5jiHnamtro66uLt9h9N49l8C+HXC1R2+Z2cCTtDIianuql7MWifWD6vnw8/8X9r8BY47P\ndzSdO7QPdm6AN1+Ft9an24bkU4J3nJEs2DWtFqrmwMhx+Y7YzPqZE0khq3lf8rnx1/CuS/IXx8E9\nGUmiw7Zv+5F1x0yGSSfBSedDWzNsWQkv/TQ9KaiYCVNrYercJMFMeRcUlw7wDZlZf3IiKWTvOANK\nxwxMIjmwq/NE8ear0PjGkXXLq5JkMePC5HPSyelnDYwoP/q7G9+C3z8HW56DzXXw8n/B899LzpWM\nhKrTk6SwmmOZAAAMI0lEQVTSvk2sTlozZjYoOJEUsuJSOGFe/8271fjW0Umiff9Ah2HG46YmyWHW\nh9MkcdLhZFE2pnc/d/QkeOcHkw0gAna9DlvqkuSyZSXUfQdWfDM5P2rS4cdhU89MBh6MOa7v929m\nOeFEUuhqFsBj1yed7mMnd183AhrfPDpJtG8Hd2VUFoyfniSG2YsOJ4rjTk5aBKWjcndPEkw8MdlO\n+6OkrLUFdryYJJX27RePkQzOAybWHG6xTKuFynfnNkYzy5oTSaGrzugnOe3jSbLYt6OTx1CvJp3c\nh/YcvlZFabI4Cd79iSNbFhNOTGYaLhTFJVD1nmSr/WxSdmhvssjX5roksby+Alb/KDlXVJL0r0w9\nM+1zOROOPwWKBuNkDWaDm4f/FrrWFvjnahhbkTxSemsDNO07fF7Fyb/sj3j8lPZZTDgBSsryFnpO\n7N12ZKtly3OHk2dZOUw9I6O/pRbGVXX/fWbWJQ//HSqKS2DuYnjlkaST+8T5R/ZXTDhheI16Kq9M\n+m1mfTg5bmuDN+vTpJK2XH5zczJiDKD8HckIsfb+Fg9BNut3bpHY0NN8MJmnrP2R2Ja65PEf4CHI\nZtlzi8SGr9KRSQtkWsaffw9BNssZt0hseOpsCPLvn4eWA8l5D0E2c4vErFsegmzWb5xIzNod8xDk\n2sMJxkOQbRhyIjHrzojyZPLM6vmHyzoOQX7hh1D37eSchyDbMOREYtZbHoJsdgQnErO+KiqCilOS\nbc4VSVlnQ5C7mgV5Wi1Mnu0hyDZo5TSRSFoI/DvJKod3RsQNHc4rPX8xyQqJV0XEc91dK2kScD9Q\nTbJC4mURsTOX92HWax6CbMNIzob/SioGXgYuBDaTrMN+RUS8mFHnYuALJInkbODfI+Ls7q6V9DXg\nrYi4QdJ1wMSI+LvuYvHwXytIPQ1BHn3ckYll6pnJTMpmA6QQhv+eBdRHxPo0oPuARcCLGXUWAfdE\nks1WSJogqYqktdHVtYuA89Pr7wZ+AXSbSMwKUrZDkF95FA9BtkKWy0QyFdiUcbyZpNXRU52pPVw7\nJSK2pvvbgCn9FbBZ3h3LEOSJ1cmnWWc+8m9w4jk5/RGD+k9fRISkTp/NSVoCLAE44YQTBjQus37V\n0xDkN1/l7RaLWUdlo3P+I3KZSLYA0zOOp6Vl2dQp7eba7ZKqImJr+hhsR2c/PCJuB26HpI/kWG/C\nrCB1HIJslke5fAX3WWCGpBpJZcDlwLIOdZYBi5WYB+xOH1t1d+0y4Mp0/0rg4Rzeg5mZ9SBnLZKI\naJF0LfAIyRDeuyJijaSl6fnbgOUkI7bqSYb/fra7a9OvvgF4QNLngNeAy3J1D2Zm1jPP/mtmZp3K\ndvivZ5czM7M+cSIxM7M+cSIxM7M+cSIxM7M+cSIxM7M+GRajtiQ1kAwVbnc88EaewjlWjnlgOOaB\n4ZgHTl/iPjEiKnqqNCwSSUeS6rIZ0lZIHPPAcMwDwzEPnIGI24+2zMysT5xIzMysT4ZrIrk93wEc\nA8c8MBzzwHDMAyfncQ/LPhIzM+s/w7VFYmZm/WRYJRJJCyWtk1SfrvdeECRNl/SEpBclrZH0l2n5\nJEmPSnol/ZyYcc3fp/exTtKH8hh7saTfSvrpYIg5Xc75R5JekrRW0jmDIOYvpn8uVku6V9LIQoxZ\n0l2SdkhanVHW6zglnSnphfTcTZI0wDH/S/rnY5WkhyRNKPSYM879jaSQdPyAxhwRw2IjmY7+VeAk\noAz4HTA733GlsVUBc9P9cuBlYDbwNeC6tPw64J/T/dlp/COAmvS+ivMU+18DPwB+mh4XdMzA3cCf\npvtlwIRCjplk2ekNwKj0+AHgqkKMGXgfMBdYnVHW6ziBZ4B5gID/A1w0wDH/IVCS7v/zYIg5LZ9O\nsvTGa8DxAxnzcGqRnAXUR8T6iGgC7gMW5TkmACJia0Q8l+7vBdaS/AJZRPKLj/TzknR/EXBfRByK\niA0k67mcNbBRg6RpwIeBOzOKCzZmSeNJ/hJ+GyAimiJiVyHHnCoBRkkqAUYDv6cAY46IXwJvdSju\nVZxKVj0dFxErIvltd0/GNQMSc0T8LCJa0sMVJCu0FnTMqW8A/50j110ekJiHUyKZCmzKON6clhUU\nSdXAGcDTwJRIVowE2AZMSfcL5V7+jeQPbltGWSHHXAM0AN9JH8fdKWkMBRxzRGwB/hV4HdhKsoro\nzyjgmDvobZxT0/2O5fnyJyT/WocCjlnSImBLRPyuw6kBiXk4JZKCJ2ks8GPgryJiT+a59F8NBTPE\nTtJHgB0RsbKrOoUWM8m/7OcCt0bEGcB+ksctbyu0mNM+hUUkSfAdwBhJn86sU2gxd2WwxNlO0peA\nFuD7+Y6lO5JGA/8A/FO+YhhOiWQLyTPEdtPSsoIgqZQkiXw/Ih5Mi7enTVDSzx1peSHcy3nAxyRt\nJHlMeIGk71HYMW8GNkfE0+nxj0gSSyHH/EFgQ0Q0REQz8CBwLoUdc6bexrmFw4+SMssHlKSrgI8A\nn0oTIBRuzCeT/EPjd+nfx2nAc5IqGaCYh1MieRaYIalGUhlwObAszzEBkI6W+DawNiK+nnFqGXBl\nun8l8HBG+eWSRkiqAWaQdJwNmIj4+4iYFhHVJP8tfx4Rny7wmLcBmyTNTIs+ALxIAcdM8khrnqTR\n6Z+TD5D0oRVyzJl6FWf6GGyPpHnp/S7OuGZASFpI8sj2YxHRmHGqIGOOiBciYnJEVKd/HzeTDN7Z\nNmAx52pkQSFuwMUkI6JeBb6U73gy4ppP0uRfBTyfbhcDxwGPA68AjwGTMq75Unof68jhCJEs4z+f\nw6O2CjpmYA5Ql/63/gkwcRDE/GXgJWA18J8kI3AKLmbgXpJ+nGaSX2afO5Y4gdr0Xl8FbiZ9cXoA\nY64n6Vdo/7t4W6HH3OH8RtJRWwMVs99sNzOzPhlOj7bMzCwHnEjMzKxPnEjMzKxPnEjMzKxPnEjM\nzKxPnEhsyFEyw+/Vx3jt8szZXruo8xVJHzy26Lr93qsk3dxDnfMlndtPP+83/fE9Zk4kNhRNADpN\nJOnEh12KiIsjmcixuzr/FBGP9SG+vjif5M32PouIfvkeMycSG4puAE6W9Hy6tsT5kn4laRnJm+xI\n+omklUrW+VjSfqGkjZKOl1StZL2SO9I6P5M0Kq3zXUmfyKj/ZUnPpWs7zErLK5Ssv7EmnRzytcw1\nIjJ+3mclvSzpGZJpZ9rLPyrp6XRyycckTUkn9FwKfDG9twWd1evkZ7xL0jPpNaskzUjL96WfX0nP\nPS9pi6TvpOWfzrjuW5KK++X/jg09A/XWqzdvA7UB1Ry5vsT5JBM01mSUTUo/R5G83XtcerwROD79\njhZgTlr+APDpdP+7wCcy6n8h3b8auDPdvxn4+3R/IcnMBcd3iLOKZAqUCpK1UZ4Ebk7PTeTwUth/\nCtyY7l8P/G3Gd3Rar8PP+Q+SOaNIf0772ib7OtSbALwAnAmcCvwvoDQ9901gcb7/33orzK3bZr7Z\nEPJMJOsxtPsLSZem+9NJ5iB6s8M1GyLi+XR/JUly6cyDGXU+nu7PBy4FiIj/krSzk+vOBn4REQ0A\nku4HTknPTQPuTyc6LCNZ3Koz2dR7CviSkvVjHoyIVzpWSOdb+h7w9YhYKelakoTybHKKURyecNHs\nCH60ZcPF/vYdSeeTzKp7TkScDvwWGNnJNYcy9luhy394HcqiTm/9B0nr5N3A57uIL6t6EfED4GPA\nAWC5pAs6+Z7rSWZG/k56LODuiJiTbjMj4vq+3JANXU4kNhTtJVmyuCvjgZ0R0Zj2aczLQQxPApcB\nSPpDkkdQHT0N/IGk45QsI/DJDjG2T+t9ZUZ5x3vrqt7bJJ0ErI+Im0hmeH1Ph/MfJUmsf5FR/Djw\nCUmT0zqTJJ3Y+a3acOdEYkNORLwJPClptaR/6aTKfwElktaSdMyvyEEYXwb+UNJqkgSxjSQJZMa5\nlaQl8BRJ4lmbcfp64IeSVgJvZJT/L+DS9s72buplugxYLel54DSSZVUz/TXJ6njtHetfiYgXgX8E\nfiZpFfAoSZ+O2VE8+69ZDkgaAbRGRIukc0hWZZyT77jMcsGd7Wa5cQLwgKQioAn4szzHY5YzbpGY\nmVmfuI/EzMz6xInEzMz6xInEzMz6xInEzMz6xInEzMz6xInEzMz65P8CJYJrgpRNAKIAAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112268208>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(nb_sizes, nb_errs, label='navie bayes')\n",
    "plt.plot(svm_sizes, svm_errs, label='svm')\n",
    "plt.xlabel('training set size')\n",
    "plt.ylabel('error')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SVM outperforms NB at each training set size"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
